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RESEARCH FRAMEWORK 

A distinction has been made between learning from computing and learning with 
computing (Jonassen, 1994). According to Jonassen, learning/ram computing refers to situations 
in which computers are tutors. Computers "direct the activities of the learner toward the 
acquisition of pre-specified knowledge or skills" (p. 4). Learning with computing, on the other 
hand, emphasizes the use of computers as cognitive tcols to extend human minds and help 
learners to construct their own knowledge. In this paper, we will examine one way of learning 
with interactive multimedia/hypermedia technology: Engaging students in designing multimedia 
programs. 

In a learner-as-designer environment, learners, instead of merely receiving information 
from computers, become an intellectual partner with the technology and engage in a constructive 
learning process (Salomon, Perkins, & Globerson, 1989). The emphasis is on using multimedia 
tools to assist in processing information meaningfully and in integrating new knowledge with 
prior knowledge. As designers, learners are given the opportunity to be creative and pursue 
actively their own intended goals. According to Seymour Papert, "better learning will not come 
from finding better ways for the teacher to instruct but from giving the learner better 
opportunities to construct" (1990, p. 3). Because of its nonlinear and associative characteristics 
as well as its use of various media, interactive multimedia is considered to be capable of assisting 
information presentation, representation, and construction (Nelson, & Palumbo, 1993) and 
capable of facilitating this leamer-as-designer process (Jonassen, 1994; Lehrer, 1993). 

Designing interactive multimedia programs is, however, a complicated and challenging 
task. This designer role calls for many critical thinking skills. Sixteen major thinking skills have 
been identified that form five categories: (1) project management skills, (2) research skills, (3) 
organization and representation skills, (4) presentation skills, and (5) reflection skills (Carver, 
Lehrer, Connell, & Erickson, 1992). Each of these skills has its own place in the entire 
development process, and is needed for producing a successful program. The development 
process, from the inception of an idea to the finished product, involves not only exercising the 
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aforementioned thinking skills but also learning various multimedia tools, working wdth team 
members, working against a deadline and, more importantly, making a strong commitment. As 
the practice in the real world suggests, multimedia development relics heavily on various talents 
in a team (McDaniel & Liu, 1996). Whether a team member is a programmer, a graphic artist, a 
designer, or a manager, collaboration among team members is crucial. Although each member 
will have his/her distinctive role, the success of a multimedia program depends on constant 
communication and understanding between team members and their working together to reach 
the goal. Such collaboration and group interaction provides a concrete and meaningful context 
for enhancing cognitive development through social negotiation (Lehrer, Erickson, & Connell, 
1994). 

Several research studies on examining learner's as multimedia designers show some 
encouraging results. Lehrer, Erickson, and Connell (1994) reported a study in which ninth- 
graders created hypermedia presentations on America history for other students to use. Their 
study showed that the designing process helped students to inter' lize various design skills. 
Students reported increases in mental effort and involvement, interest, planning, collaboration, 
and individualization (Lehrer, Erickson, & Connell). Spoehr (1993) found that designing in 
hypermedia could help students develop more complex knowledge representations and assist the 
development of their thinking skills. 

Literature on motivation and classroom learning has shown that motivation plays an 
important role in influencing learning and achievement (Ames, 1990; Dweck, 1986). Beichner 
(1994) described a study in which junior high schools students used a multimedia editing 
program to produce information screens for zoo visitors. The result showed that students were 
highly motivated and often spent extra time working on the project. The study by Lehrer et al 
(1994) found that students' time on-task increased significantly over the course of successive 
design projects, and the design process helped them to make the transition from receivers of 
knowledge to authors of knowledge. 
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RATIONALE OF THE STUDY AND RESEARCH QUESTIONS 

Designing a multimedia program integrates a variety of activities, incorporates a wide 
range of thinking skills, and "taps a diverse set of skills" (Carver et al, 1992, p. 388). The 
comprehensive nature and extensiveness make it a unique learning opportunity, an opportunity 
that can lead to the development of highly valued mental skills important for learning. The 
instructional benefits of such a learning-with-technology approach are obvious. However, 
research also indicates that implementing a learner-as-designer environment is not an easy task. 
Indeed, any instructional situation involving such a process requires some extent of "adventurous 
teaching" (Sheingold, 1991). One important issue for designing such an environment is the 
extent to which the design knowledge, design discussions and design skills are made explicit to 
the learners. In the study by Carver et al., it was found that explicit discussions of lesign skills 
organized by the teachers were far from sufficient and, therefore, students lacked an 
understanding of the overall design process. Their data revealed that students, though acquiring 
individual skills, continued to need help from adults in putting together a project. They 
concluded that "the challenge to be met here and elsewhere is the design of learning 
environments that allow students to develop their own interests, yet provide students with 
comprehensive skills that can be applied in a wide range of potential contexts" (Carver et al, p. 
402). They point out that "...explicitness and practice are the key ingredients for success" (p. 
401). 

In an effort to meet this challenge, we used the cognitive apprenticeship model (Collins, 
Brown, & Newman, 1989) to construct a learner-as-designer environment simulating a real- 
world multimedia production house. Students were given with the opportunity to work with a 
client, a local Children's Museum and/or their school. They were provided with a four-phase 
multimedia design model: (1) Planning, (2) Design, (3) Production, and (4) Evaluation and 
Revision. This design model incoiporated many steps in Lehrer's instructional model (Lehrer, 
1993) and many design steps used by local multimedia developers (McDaniel & Liu). (A 
detailed explanation of the model was provided in method section.) 
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In this learner-as-designer environment, the explicitness of the design process was 
implemented in several ways: (1) direct instruction about the design process; (2) a simulated 
multimedia production environment in which each student had chosen or was assigned a role of 
being a programmer, a graphic artist, a project manager or all three; (3) direct interaction with the 
clients of the programs; and (4) direct communications with multimedia experts. The four-phase 
design model was presented to students during the first week of the class. Students were then 
engaged in various design activities, ranging from the brainstorming of topics for development to 
visiting and making observations at a children's museum to get a sense of what would worked for 
young children; from evaluating commercial multimedia CDs to learning various multimedia 
tools. Students had the opportunity to listen to several guest lectures by local multimedia experts 
on their experiences of designing multimedia programs. They also participated in field trips to 
local multimedia production houses. In addition, a graduate student and two people from a 
children's museum, one of them an experienced multimedia educator and the other a graphics 
artist having years of experience in multimedia production, worked closely with the students 
throughout the semester. Student learning was scaffolded in the following ways: (1) explicit 
design instruction; (2) learning multimedia tools; (3) coaching by the teacher, the museum 
representatives and the graduate student; and (4) interaction with various multimedia experts. 
Assessment used in the study (questionnaires and interviews) provided additional scaffolding for 
the studciits (Lehrer et al.). This collaborative design environment was meant to have an added 
value in that it provided an opportunity for students to design multimedia programs for a real 
audience and to work with multimedia experts. 

The purpose of this study was to examine the learning benefits of this learner-as-designer 
environment and its impact on the students. Specifically, the following two research questions 
guided this study; 

(1) What is the effect of this "learner-as-designer" environment on students' motivation? 

(2) How does such an environment contribute to students' learning of design knowledge? 
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METHOD 

Sample 

Two classes of students from a high school in a mid-sized Southwestern city participated 
in this study. The school is an inner c/ . igh school with a large minority student population 
consisting of: Caucasian = 31%, Hispanics = 58%, African Americans = 11%. About 45 percent 
of the students are from low income families, 10 percent are qualified as Limited English 
Proficiency (LEP) students, and approxiniately 60 percent of the population are considered to be 
at-risk for not finishing high school. These two participating classes reflected the demographics 
of this minority dominant school. 

The treatment class consisted of 24 10th-, 11th-, and 12th- graders from a computer class 
where the teacher engaged students in designing multimedia programs. Se^ enty-one percent of 
the treatment class were minority students (42% = Hispanics, 17% = African Americans, 12% = 
Asian Americans or American Indians), and 16% of the class were considered to be learning 
disabled (LD) students. The control group consisted of 22 1 1th- and 12th- graders from an intact 
computer class in which various computing tools were taught (72% minority: 54% = Hispanics, 
9%= African Americans, 9% = Asian Americans). Although there were 24 students total in the 
treatment class and 22 total in the control class, only those who had completed aJi the 
instruments given were included in this study — sixteen from the treatment class (14 male, 2 
female) and fifteen (9 male, 6 female) from the control class. 

A treatment class and a control class were used to understand the effect of the "learner-as- 
designer" environment on students' motivation. Students in these classes were similai' in age and 
ethnic background. Both classes were involved in computing. 

A Learner- as-Designer Environment 

The treatment was a learner-as-multirredia-designer environment. The students in the 
treatment class were told, at the beginning of the semester, that their goat was to develop a 
multimedia program for other people to use. The class met everyday for 90 minutes for 18 
weeks, a total of 135 classroom hours. The organization of the class followed the four-phase 
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multimedia design model mentioned eaiiier. Students were given the opportunity to work with 
the local children's museum. During the planning phase, the four-phase design model was 
presented to the students. Lectures on how to make a successful multimedia program were given 
by local multimedia experts. Students were engaged in brainstorming sessions and making 
decisions on what to create (the content), whom to create for (the audience), and how to develop 
the program (the process). Many topics were proposed for inclusion. After discussio.. and 
negotiation considering their own interests, the desires of the museum clients, and time factor for 
completing the project, students formed four teams. One team decided to work on a 
demonstration of an electronic yearbook for the school, and the other three decided to create a 
virtual museum for the Children's Museum with three different topics (physics, dinosaurs and 
history). All groups evaluated some commercially available multimedia CDs and noted the 
features that they liked and the ones that did not work well. This exercise was intended to help 
students develop their critical Judgment and learn from both the good and bad features of the 
commercial products. They also visited the Children's Museum to get a sense of which exhibits 
were effective for young children and which exhibits were not. They used this information as the 
stalling point for their brainstorming sessions. 

In the design phase, students were engaged in defining and refining the topic, subtopics, 
the age level of the audience their programs would target at and strategies to use for presenting 
information. Each team created a flowchart or a storyboard, detailing how each screen was 
related to others and the overall structure of the program. Each team member chose or was 
assigned the role of a researcher, a graphic artist, a programmer, a project manager, or an 
animator, depending on his or her background and the program's requirements. Some students 
assumed more than one role, while some took on different roles at different times. The 
storyboard/flowchart was realized on the computer during the production phase. They learned 
some of the state-of-art multimedia tools including Adobe Photoshop , and Macromind Director . 
0. qris Works and ^ perPaint were also among the programs used. Although some stude.its had 
knowledge of Ci a i ig Work;? and g_up,QfPqip.t from previous computer classes, few knew 
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Photoshop or Director . None had participated in a similar learning context before this class. 
Students drew all the graphics, recorded their own voices and created ail the animation from the 
scratch, so as not to violate copyright laws. Then all the pieces were assembled into the final 
programs. Like practice in the real world (McDaniel & Liu, 1996), the evaluation and revision 
phase was intended to be a dynamic and interactive process in that the program.s were being 
evaluated on a continuous basis by team members, members from other teams, and students in 
other classes. Revisions were made immediately. 

The students had access to three Pov/er Macintosh computers (received about two months 
after the semester started), eight Mac LCills, one Mac 575, one Mac 520, a Cannon Xapshot 
camera, and a color scanner. The computer equipment was, however, spread in two separate 
rooms due to the constraints of physical layout of the classrooms. The two main programs used. 
Director and Photoshop , could only run on the tliree power Macs, the Mac 575, and two of the 
Mac LCIIIs because of their large memory requirements. Students in the four teams, therefore, 
needed to share the available computer resources. 

A multimedia educator as well as a graphics artist from the Children's Museum worked 
closely with the three museum ‘earns. One graduate student from a local university who had 
experience in multimedia development worked closely with the CD-yearbook team. The teacher 
acted as the project manager for the entire class overseeing the various phases of the design and 
production process. During the semester, students made trips to local multimedia development 
companies, multimedia users group meetings, and a mmltimedia graduate class at a university to 
learn and share their experiences. At the end of the semester, students had an open house to 
demonstrate their programs to parents, administrators, and the local industry. 

The control group came from an intact computer application class, in which various 
computer-related topics were taught. The topics covered included word-processing, database, 
spreadsheet, telecommunication, and desktop pubiishing. The students in the control group had 
access to 30 286-PCs, 1 386-PC, 4 486-Dell computers, I color printer and I black & white 
scanner. 




d 



Learners As Multimedia Designers - 9 



Data Sources 

The data sources for this study consist of both quantitative data (instruments to the 
treatment and control groups) and qualitative data (observations and interviews from the various 
perspectives, and analysis of students' multimedia programs). The triangulation of the data helps 
to provide a better picture of the environment under study. 

Motivation Questionnaire. To assess students' motivation of working in this 
environment, a 26-item questionnaire was used. This questionnaire addressed five areas of 
motivation, and was taken from the Motivated Strategies for Learning Questionnaire (MSLQ) by 
Pintrich, Smith, Garcia, & McKeachie (1991). The areas addressed are (1) intrinsic goal 
orientation (4 items), (2) extrinsic goal orientation (4 items), (3) task value (6 items), (4) control 
of learning beliefs (4 items) and (5) self-efficacy for learning (8 items). Examples from this 
instrument are "In a class like this, I prefer course material that really challenges me so I can 
learn new things," and "I am confident I can learn the basic concepts taught in this course." A 
seven-point Likert scale was used with 1 being "not at all true of me" and 7 being "very true of 
me." This instmment was given both to the comparison and treatment groups at the beginning as 
well as at the end of the semester. 

Project Design Questionnaire. A 60-item questionnaire on various design activities 
developed by Lehrer, Erickson, and Connell (1994) was used.' It uses seven-point Likert scale 
with one being "not describing me at all" and seven being "describing me very well." These 60 
items address nine categories of design: planning (8 items), searching information (8 items), 
presenting information (8 items), connecting ideas (5 items), audience (3 items), collaboration (8 
items), mental effort and involvement (8 items), interest (8 items), and individualization (4 
items). Examples of this questionnaire are "I make sure I understand all of the topics before I 
start putting my presentation together," "I often use illustrations and other pictures in my 
projects," and "Overall, I feel positively about working with others on a project." This 



*The original list consists of 66 statements. Six items were not included because of their irrelevance to this study. 
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questionnaire was administered to the treatment group as a pretest and as a posttest. The 
Cronbach alpha reliability coefficient for this questionnaire was .97. 

Design Task Listing and Ranking. Students were asked to list the tasks they needed to do 
to create their multimedia programs (Lehrer et al, 1994). For each task, they were asked to list 
what had helped them to accomplish the task and what had hindered them. They could list as 
many tasks as they wanted to. For example, one student listed "storyboarding" as a task needed 
for the project. "Working in a group" v/as listed as a helpful strategy for completing the task and 
"taking a lot of production time" was listed as an obstacle for the task. After completing the 
design task listing, students were given a list of 18 tasks relevant to their project development 
and asked to rank the tasks according to their relative importance (Lehrer et al). Examples of the 
given tasks are "Think about the best way to present an idea," "Make video clips," and "Discuss 
with your team what information to include." These two instruments were given to the treatment 
group at the end of the semester. 

Observations and Interviews. Observations of the class activities for the treatment group 
were made throughout the semester. Observations by the teacher, people from the museum, and 
the graduate student were also obtained through the interviews. The observations focused on the 
tasks students were performing, their time on-task, communication among the team and with the 
teacher and the clients. During the semester, interviews were conducted six times with each of 
the teams on their design and thinking process.^ Thirty percent of the interviews with each team 
was unstructured, allowing students to take the interviews into unanticipated directions. Seventy 
percent of the interviews was structured. The interview questions concentrated on the following 
aspects: (1) the tasks they performed during that particular period of time, (2) their time on-task 
in school and out of the school, (3) the various design skills they used and talked about, (4) work 
among the team members - division of work, sharing among members, problems and conflicts, 
(5) their feelings, and attitudes, (6) the tools and techniques they learned, and (7) what they had 

^ At first, we wanted to videotape the class activities. However, after an initial videotaping, we realized that 
videotaping technique would not work well in this context, because students were too nuibile, moving from one 
room to the other constantly to share the computer resources among the teams. Focusing on one team with one 
video camera at a given period was not possible. 
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accomplished since the last time interviewed. Students were also interviewed about their 
thoughts and feelings toward specific activities such as field trips to local multimedia companies, 
talks by guest speakers, and evaluation of their products by peers. The same quesbons were 
asked with all four teams to provide commonalty across the interviews. Follow-up questions 
were asked whenever clarification was needed or more information was desired. The questions 
on many of the aspects mentioned were repeated over several interview's to provide a better and 
more complete picture ovei an extended period of time. Respective interviews were also 
conducted with the classroom teacher, the people from the Children's Museum, and the graduate 
student, once at mid-semester and once at the end of semester. The time for each interv.’ew 
ranged from 10 minutes to 45 minutes, depending on the available schedules of the interviewees. 
These interviews produced five and half 90-minute tapes, a transcript of 165 single-spaced pages. 

Evaluation of Students' Programs and Their Teclvvcal Skills. The programs that the 
students developed were evaluated on the following categories: (1) content -- complexity of the 
content, and appropriateness of the content to the target audience, (2) stmcture - links between 
various concepts, (3) screen design — how well the message was understood through the use of 
color, font, and icon; screen layout, and navigation, (4) use of media — the use of graphics, 
animation, audio and video, and (5) creativity — originality of the content and design, originality 
of the graphics, video and animation. At the end of the semester, each team was asked to 
demonstrate their programs, explain the various components, rationale for using certain media, 
and procedures for creating certain segments of the program. 

Procedures 

This was a semester-long study, lasting approximately 18 weeks. At the beginning of the 
semester, both the treatment and control groups were given the motivation instrument. The 
treatment group was also given the project questionnaire. Each class then was engaged in its own 
instructional activities. The students in the treatment group were observed throughout the 
semester and each of the four design teams was interviewed approximately every three weeks for 
a total of six times. At the end of the semester, the motivation instrument was again given to 
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both the treatment and control groups. In addition, the treatment group was also given the project 
questionnaire, design task listing, and ranking forms. Students' programs were collected and 
evaluated. 

Analysis of the Data 

To determine the effect of this "leamer-as-designer" environment on students’ motivation 
(research question 1), five two-factor mixed ANOVAs were run with the grouping (treatment vs. 
control) as a between-subjects independent variable and the data collection points (pre vs. post) 
as the repeated measure independent variable. The dependent variable for each respective 
analysis was the pre and post scores for intrinsic goal orientation, extrinsic goal orientation, task 
value, belief and self-efficacy. 

To find out how such an environment contributed to students' learning of design 
knowledge (research question 2), nine one-way ANOVAs with repeated measure of the pre/post 
data collection points were conducted. The dependent variable for each respective analysis was 
the pre and post scores of each of nine categories in project questionnaire: planning, searching 
information, presenting information, connect' ng ideas, audience, collaboration, mental effort, 
interest and individualization. Data from Design Task Listing and Ranking forms were analyzed 
descriptively. 

The interview data was first transcribed, then chunked, coded and categorized following 
the guidelines by Miles and Huberman (1994). A two-level scheme, as described by Miles and 
Huberman, was used. At a more general level, a start list of codes were constructed using the two 
research questions as a guide. At a more specific level, codes generated directly from the 
interview data were nested in the more general codes. During the coding processing, the codes 
were refined, revised, and new codes for emerging themes were added. Patterns from the data 
were extracted and relationships between coded segments were studied. The data was then sorted 
into categories and sub-categories according to their common themes and shared relationships. 
Each researcher coded the data independently. Disagreements between the researchers were 
discussed and resolved after reexamination of the data until a .95 interrater reliability was 
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achieved. Observation notes were also written up and analyzed. These qualitative data were used 
to provide more detailed descriptions and substantiate the results from the statistical analyses 
mentioned above. 

RESULTS 

Findings from Quantitative Data Sources 

Motivation 

The results of the two-factor mixed ANOVAs indicated that there were significant two- 
way interactions between the grouping (treatment vs. control) and the data collection points (pre 
vs. post) for intrinsic goal orientation, task value and self-efficacy: F(l,29)intrinsic = 6. 

.05; F(l,29)task = 2.84, p< .05; F( 1 ,29)self-efficacy = 9.71, p< .01 (see Table 1). The treatment 
group increased its intrinsic scores from pre to post whereas the control remained the same; the 
treatment group increased its task value scores from pre to post whereas the control group 
decreased its scores; and the treatment group increased its self-efficacy scores from pre to post 
whereas the control group remained about tne same (see Figure 1). In other words, the treatment 
group had significantly increased its scores of intrinsic goal, task value, and self-efficacy from 
pretest to posttest, and the differences between pretest and posttest for the treatment group were 
significantly greater than those for the control group. The two groups, however, were not much 
different in scores of extrinsic goal orientation and control of learning beliefs: F( 1 ,29)extrinsic = 
.83,/?= .37; F(l,29)belief = •25,/7=.62; (see Table 1). 



Insert Table 1 here 



Insert Figure 1 here 



Design Skills 
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The one-way ANOVAs with repeated measure analyses showed that there were 
significant differences between the pretest and posttest scores for the treatment group in the 
following categories: (1) planning: F(l,15) = 8.73, p <.01; (2) presenting information: F(l,15) = 
9.12, p <.01; (3) collaboration: F(l,15) = 12.26, p <.01; (4) mental efforts and involvement: 
F(l,15) = 1 1.78, p<.0l\ (5) interest: F(l,15) = 6.67, p<.05 (see Table 2). That is, the posttest 
scores in these five categories were significantly higher than the pretest scores. However, the 
differences between pretest and posttest scores were not significant for (1) searching information: 
F(l,15) = .3,p = .59; (2) connecting information: F(l,15) = 3.21, p = .09; (3) audience: F(l,15) 
= 2.82, p = .1 1; and (4) individualization: F(l,15) = .06, p = .8 (see Table 2). 



Insert Table 2 here 



Students listed the design tasks that, in their opinions, were needed to complete their 
projects, and helpful strategies as well as obstacles for the accomplishment of the tasks. The 
result of the descriptive statistics showed that the more frequently mentioned tasks included 
(arranged from the more frequent to the less frequent): (1) doing storyboards; (2) researching; (3) 
programming; (4) brainstorming; and (5) creating graphics (see Table 3). The tasks that were 
mentioned least often included (also arranged from the more frequent to the less frequent): (1) 
scanning pictures; (2) digitizing voices; (3) making backgrounds; (4) digitizing music; (5) taking 
photos; (6) testing (see Table 3). The more frequently mentioned tasks are those that require 
more thinking skills whereas the least frequently mentioned tasks are more mechanically 
oriented. Interestingly, the task "testing" was only mentioned once, and no examples were given. 

Insert Table 3 here 



Students were given 18 tasks relevant to multimedia production and asked to rank the 
tasks according to their relative importance. Students' ranking revealed their understanding of the 
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importance of the tasks in designing and producing multimedia programs. The result of task 
ranking corresponded quite well with task listing assessment discussed above. Of the 18 tasks, 
the six tasks that received highest ranking for their importance included: (1) creating storyboard 
(Mean = 3.9, the lower the score, the higher the importance); (2) taking notes on what you read 
(Mean = 4.6); (3) discussing with the team what information to include (Mean = 4.7); (4) 
brainstorming the best way to present an idea (Mean = 5.0); (5) learning multimedia tools (Mean 
= 5.6); and (6) going to the library and finding relevant information (Mean = 5.6). Students 
regarded these six tasks as most important when compared to others. The following six tasks 
received lower ranking for their importance: (1) making very colorful graphics/illustrations 
(Mean = 10.0; the lower the score, the higher the importance), (2) scanning photos (Mean = 
11.0); (3) making animation (Mean = 1 1.0); (4) making video clips (Mean = 12.0); (5) making 
sounds (Mean = 12.0); and (6) getting someone to try out the programs (Mean = 13.0). These six 
tasks, in students' opinions, are least important when compared to others. It is clear that the most 
important tasks for the students ai’e those with greater emphasis on higher order thinking skills 
whereas the less important tasks are those more of a mechanical nature. It is noted that once more 
students did not give much attention to the task of getting others to evaluate their programs. 
Findings from Qualitative Data Sources 
Time Spent. 

Class periods typically consisted of whole class activities and group work. Examples of 
whole class activities were brainstorming sessions, instructions from the teacher and guest 
lectures from local multimedia experts, field trips to local multimedia companies and instruction 
on multimedia tools. Group work referred to students working in their own teams, storyboarding, 
learning multimedia tools, and creating their projects. Classroom observations and interview data 
revealed that beginning from the mid-semester, the time spent on the projects in class as well as 
out of the class, and time on task had noticeably increased (see Table 4). 



Insert Table 4 here 
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Interest. 

Students' interests in the multimedia production process were reflected in several aspects. 
In addition to the increase of time spent and time on task, the data showed that students liked the 
class and the projects they were working on. Statements such as "It [working on the project] is 
good for our education," "The experience is important. And it could help us get jobs," "This is 
just something that I like to do. [It] will be real help in the long run," "It [the class] prepares you 
more for the future" occurred frequently throughout the interviews and with all groups. Students 
self-reported that they hardly skipped the class. The following was a sample of responses from 
the students when they were asked "How do you compare this class to the other classes you are 
taking?" 

This class is different. We let our own curiosity help us to learn. 

I don't think that I have ever taken a class like this one. Computer class[es], they 
didn't teach you anything like this.... We didn't learn anything compared with 
this. A lot of stuff we're learning now. 

I'm making a lot better grades than in some of my other classes. It's a lot of fun. 

It [this class] is more related to the future. Something you'll actually use. 

It's more at your own pace. It's different in that the teacher is not always 
teaching you, you teach the teacher and he helps you and you help him so 
everybody becomes a teacher. 

I think it [this class] is better and you can get more things done. People are 
having the chance to use their abilities and their knowledge to help other people 
out. 

Self-Efficacy 

Findings also indicated that the students' confidence in themselves had significantly 
increased. At the beginning, students were not sure if they could pull together as a group to 
accomplish the project by the deadline. They exhibited their confidence more clearly after 
making a field trip to a local multimedia company which had won a number of national media 
awards for their multimedia CD titles. The purpose of the trip was for students to show their 
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work and to learn from the multimedia professionals. The trip was purposefully arranged during 
the second half of the semester when students had completed a significant amount of their 
projects and had a first-hand experience of multimedia production. The interviews conducted 
after this trip revealed students' high respect for multimedia professionals they met, a longing to 
be a part of that professional production team, confidence in their own abilities, and 
determination for the future (see Table 5). 



Insert Table 5 here 



The statement "I want to work there" was echoed by most of the students. Their 
excitement and determination in getting multimedia production experience, and their confidence 
in their own capabilities were observed by the teacher, as well as the museum people. One of the 
museum people said: 

The kids were very excited about what they had seen. So when asked if they 
wanted to work there, a lot of hands went up. I do. I do. ... These kids were 
showing their stuff and they [the professionals] were watching and they were very 
interested in it. They [the professionals] were looking at this and that. They were 
asking each other what [software] they [students] were using. Some expressed a 
desire that they could have had this [the experience of producing multimedia CD] 
when they were in school, then they would have known what to do with 
themselves. 

Students' enthusiasm and confidence were also shown during the open house where they 
demonstrated their final products to their parents, local multimedia experts, university professors, 
and students from other classes and high schools. 

Brainstorming and Storyboarding 

The observation field notes indicated that the process of narrowing down the 
development scope due to the time constraint and limited resources was not easy for the students 
during the initial brainstorming sessions. Because this was a new experience to all the students, 
they did not realize that producing multimedia programs took a tremendous amount of time. 
Their enthusiasm was high and it was painful for them to drop some wonderful ideas they came 
up with for development. One student said, "we had too many good ideas and we couldn't use all 
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of them, because of time and computer memory. It was hard picking those few things we could 
do in one semester." The need for more time also existed with each group when it started to 
develop its own project. "We've only got an hour and half each day to work on it. The deadline is 
getting closer and there is so much to do... one student stated. "I wish we had more time. Then 
we can redo some of the graphics and add more stuff," another student commented. 

One of the interesting findings was the change in students' perception toward 

storyboarding. Each team was asked to do a storyboard for its project, mapping out each screen, 

as one would do in a professional environment. It turned out that this task was very difficult for 

the studencs to accept. The students had these comments initially: "We just do not understand 

why we need to do this [stoiyboarding]." "It takes so much time and so boring. I want to work on 

the computer." The teacher said: "It was very difficult to get them [students] to do the 

storyboards. It was like pulling teeth for some groups. But once we got onto the actual software 

itself and started to put the stuff together, we’ve had kids come in during lunch and stay after 

school." However, interestingly, during the production phase, storyboards were a common sight 

in the classrooms where students consulted them from time to time. The following is what they 

had to say about storyboarding toward the end of the semester: 

It [storyboard] helps us when we get on the computers. It also helps when we 
are explaining what we are doing to other people [other team members]. They 
can be looking at that while we are explaining it to them. 

[Do you think the storyboard helps you?] Oh, definitely. We were lost. We 
didn't know which way the program was going to go. We didn't know who was 
going to do what. Now that we have the storyboard, we can separate the 
program into different pieces and give certain people assignments like to 
lay all the text out, and to draw, or to put it together. Now we have an idea of 
what we have to do. 

It is very [necessary]. We wouldn't have known how to separate all the work and 
we wouldn't have known exactly how it was going to flow. 

When students were asked, "what have you learned from the field trip [to the professional 

multimedia development house]?," some had this to say: 

The main thing we got was we need more storyboarding, planning out better 
before we start. 

I think we should plan it out to the last detail. 
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They [the professionals] had everything planned out. I would have planned it out 
completely before I started working on it. 

Storyboarding, planning them out more thoroughly before we begin. 

Probably the brainstorming, storyboarding and everything. You don't want to go 
straight into it and start making all that stuff, [bejcause then you don't know 
what you are going to use. You want to know exactly what you need, that way 
you can get it finished as quick a,s you can. ... You could make something really 
cool but then you can't use it because you find out you don't need it. You just 
want to do things you need to do for the CD program. 

Students' perception change about storyboarding was observed by the museum people. 

One said: 

At first they balked and they didn’t want to do it [storyboarding], but then they 
understood and we brought in some storyboards from freelance jobs we were 
working on too, to shov/ them. That helped as well. So I think they understand it 
now, and they quit complaining. Their stuff is so much more directed and 
cohesive... They checked them [storyboards] a lot. It's like their blueprint. 

The teacher also reported, "Initially they did not want to do the storyboard. But then, I saw them 

grabbing their blue books [notebook containing storyboards] and referring to them 'OK, we need 

to do this or that.. What images do we need? Well, we'll go look and find out' ..." 

Teamwork 

Multimedia production is more of a team work than an individual's effort. From the 
beginning, the importance of teamwork had been emphasized and the process of using teamwork 
had been implemented. A group of students worked on one project, rather than an individual. 
Each member of a team was assigned a role by the team project manager or volunteered for a 
role because of his or her background and interests. Students' understanding of the team effort 
and their collaboration among teams were reflected in several aspects: (1) sharing in a team, (2) 
helping each other, (3) appreciating other's work, (4) taking other's suggestions, (5) deciding 
together, (6) getting along with each other, and (7) exerting peer pressure (see Table 6). It was 
clear that the students had a good understanding of the role a team should play in multimedia 
development. Although conflicts did exist, students were able to resolve them and worked 
together to accomplish the project. 
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Insert Table 6 here 



Learning Multimedia Tools 

The software that the students needed to learn were among the state-of-art multimedia 
tools. Programs such as Macromedia Director and Adobe Photoshop are the choice of 
multimedia professionals. These programs allowed students to use their imagination and create 
what they wanted. But the learning curve was very steep. Although the very basic tenets of the 
programs were provided by the teacher and the graduate student, students learned, on their own, 
many techniques and details of the programs by using the manuals and helping each other. One 
student said, "D came in and taught us the basics. Then you just keep adding on because you can 
learn by yourself." Another student said, "[Learning by] Hands on. Look at all the options you 
have. Just try to learn, try to figure it out." Helping each other within a team and often among the 
teams was a common practice. When a student learned a certain technique and people in other 
teams wanted to use this technique, he/she would share it with them. As one said, "Usually if we 
don’t know it already, [a] classmate teaches you." A student was commenting on a 3-D program 
he was using, "I figured [it] out some and got help from other students." 

Throughout the semester, the graduate student was in class almost eveiyday except for a 
couple of weeks. One museum person came to the class three times a week and the other two 
times a week. Although the teacher, the graduate student, and the museum people worked very 
closely with the students, their rotes were more of assisting and facilitating than directly 
teaching. They provided feedback and suggestions to students, and they helped students when 
they had questions. "He [the graduate student] leaves us alone to figure out the basics. But if we 
get stuck or have a question about it, we'll call him over and he will help us out. He likes to leave 
us to learn on our own," one student said. Group interaction and students' comments showed that 
students had taken on the responsibility of "figuring out ourselves." In fact, they enjoyed the 
challenge. 

Researching 
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The research aspect of the process took the forms of (1) evaluating commercial 

multimedia CDs, (2) searching for and collecting relevant information and (3) preparing the data 

for presentation in the programs. At the beginning of the semester, the class was asked to 

evaluate critically a number of commercial multimedia programs available, and discuss the 

strengths and weaknesses of each. This activity helped students to see what v/as available, the 

ideas and techniques that they could incorporate in their programs as well as what they should 

avoid. Each of the four teams had assigned researchers to collect information about their topics. 

They went the school library to search for the relevant information and wrote up what they found 

to give other team members to put on the computers. The CD yearbook team designed a survey 

to give out: "We made surveys and passed them out. It was more of a group idea, because it told 

us what the public wanted to see in our yearbook." The physics group got some help from the 

physics teacher in the school about illustrating the concept of gravity. They said that by working 

on the project for young children, they had an opportunity of reviewing what they had learned in 

physics. A student commented on their research process: 

We went to [the] libraiy and did research on a lot of topics and found pictures we 
could possibly scan or draw. Then we talked to some teachers and found out a 
little more information. After our research was finished, we went back three or 
four times to get more in-depth research. I thought I was done the first time, but I 
wasn't.... If we didn't do any research then it would have been hard to base what 
we are doing. 

Evaluation of Students' Technical Skills and Their Programs 

Students' Technical Skills 

The teacher's assessment, as well as the observation and interview data showed that 
students made good progress in learning and mastering the multimedia software used. All of the 
students had a good command of Claris Works and SuperPaint by the end of the semester. Many 
of them developed sufficient competency in using Adobe Photoshop and Macromind Director . 
A number of the students also learned on their own SoundEdit 16 for digitizing audio, Ofoto for 
scanning, and the Cannon Xapshot camera. Two students learned by themselves Virtus 
WalkThrough . a 3-D program, and Adobe Premiere for digitizing video. Students' command of 
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the multimedia tools were reflected in the programs they developed, as well as in the statements 
made by different people (see Table 7). 



Insert Table? here 



Multimedia Programs Students Developed 

Students developed four multimedia programs: (1) CD yearbook demo, (2) a history 
adventure, (3) a dinosaur adventure, and (4) a physics demonstration for five-year-olds. All of 
the programs were composed from scratch. That is, all components (graphics, audio, animation, 
video) were made by the students. The media used by all four teams included text, graphics, 
audio, and animation. Video was used only a few times as the computers that they had were not 
equipped to do any high quality digital video. The scope of these programs was quite extensive 
considering these were projects completed in one semester. For example, the CD yearbook 
demonstration consisted of five subsections: sports, clubs, school life, hang-outs, and student 
portraits. The student portraits section contained several hundred scanned photos of individual 
students. The Dinosaur program consisted of three subsections: picture show, game, and check 
test. The History program consisted of three periods of history: Pirates of the Caribbean, the Wai' 
of the Roses, and King Tut's Egypt. The simplest program of all was Physics for five-year-olds, 
which expounded upon the concept of gravity and has (1) the gravity building, (2) the clock, and 
(3) click-clack. Figure 2 shows several screens of these programs. 



Insert Figure 2 here 



DISCUSSION 

Multimedia Production and Motivation 

Interest and Involvement 
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The ANOVA results showed that the treatment group had significantly increased their 
scores of intrinsic goal orientation and task value, but not the control group. The insignificant 
result in extrinsic goal orientation for the treatment group showed that students' interest in doing 
multimedia projects was not necessarily dependent upon getting a better grade. 

The results suggested that students who were involved in designing multimedia programs 
were more motivated toward learning than those who were not. Students' interest and 
involvement in the projects were reflected in the increase of their on-task behavior and the 
amount of time spent working on the project. As the qualitative data showed, many students 
spent their lunch period, and time before school and time after school working on the project. 
Some students took work home, some came early in the morning, and some spent part of their 
spring break and Easter weekend working on the projects. Instead of being isminded by the 
teacher, students began to remind each other of being on task. They exhibited strong motivation 
to get the projects done. 

The results also indicated that students enjoyed this multimedia design experience, which 
allowed them to be creative and "let [their] own curiosity help [them] to learn." They felt that the 
experience was valuable as it prepared them for the job market. They saw relevance in what they 
were doing. It was quite obvious that the real-world implication of the task and its usefulness to 
the future became important motivational sources. 

This finding supports other studies which have found that designing multimedia programs 
is a motivational task for students and can help students to be more on task (Beichner, 1994; 
Hays, Weingard, Guzdial, Jackson, Boyle, & Soloway, 1993; Lehrer et al., 1994). The results 
showed that this learner-as-designer environment did have a positive impact on these at-risk 
students' motivation. 

Although the general pattern of time spent increased significantly throughout the 
semester, some fluctuations existed. It was observed that sometimes, particularly during the 
earlier portion of the semester, some students would be "playing" on the computers rather than 
working directly on the project. The teacher acknowledged that some software was so intriguing 
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that "some [students] would spend the whole period playing with filters [in Photoshop! 
"Playing" on the computer and figuring out what filters could do was part of the learning 
experience and could typically contribute to development of the project. However, due to time 
constraints, the students did not have the luxury to "free" play and find out all the details of the 
software. Multimedia production was a new experience for the students. The task of producing a 
multimedia program was demanding and time consuming. The regular class period of an hour 
and half a day appeared to be too short and insufficient for multimedia development. One 
semester was hardly enough for these beginners who had to learn not only the multimedia tools, 
but also all of the design concepts and skills. It became also obvious that time management was 
an important skill that these students were yet to learn. 

Confidence and Self Image 

Both the ANOVA analyses and the qualitative data have shown that students involved in 
designing multimedia programs have significantly increased their self-efficacy and have a much 
more positive image about themselves than the control group. This finding is in line with other 
studies conducted on the same topic (Beichner, 1994; Lehrer et al., 1994). 

Students' confidence in themselves was most clear both after a trip to a multimedia 
company and during the open house. A possible explanation for this was that going to a 
professional multimedia company, having their products judged by other p^^ople, and receiving 
positive feedback gave the students an opportunity to compare their work with the professionals' 
work and see how their work stood in comparison. Most of these students were average or below 
average academically. Several of them had learning disabilities, and quite a few were among the 
at-risk population. However, designing and producing multimedia programs allowed them to use 
and develop their own abilities. The design process they were engaged in was similar to what 
was used by multimedia professionals. The software they used v/ere the choices for professional 
multimedia development. Their work had received positive feedback from the professionals and 
impressed many visitors at the open house. The students realized that what they lacked was time 
and experience. If they continued to gain experience in multimedia development, they could 
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eventually reach a professional level of competency. The statement "I think the only difference 
[between ourselves and professionals] is [the] experience" reflected their thinking. This 
realization excited many students. The authentic aspect of the experience helped to increase their 
confidence and set their goals for the future. Before taking the class, many students were 
working in fast food places and had no higher goals in life. At the end of the semester, close to 
50% of the students talked about working in multimedia industries after graduation from the high 
school. In fact, four were hired by the Children’s Museum and local multimedia companies as 
summer interns or paid employees as a result of this experience. This multimedia design 
experience has made a positive influence on these students who belong to the at-risk population. 
Multimedia Production and Design Knowledge 
Importance of Storyboarding 

This leamer-as-designer experience changed students' perception tov/ard story boarding. 
Initially, many of these students wanted to get on the computers as quickly as they could. The 
task of planning out every screen and its relationship with the other screens was not only 
difficult, but was also very boring for them. However, students changed from being impatient 
and reluctant in doing storyboards to readily using storyboards in their production process. 
Complaints about doing storyboards were replaced by a desire to do better and more detailed 
ones. As the design task listing and ranking analyses indicated, storyboarding was the most 
frequently cited task the students performed and it received the highest ranking for its 
importance. Several elements of this learning environment helped to bring about this change: (1) 
the actual production experience, (2) the scaffolding by multimedia professionals, and (3) close 
working relationship with their clients (i.e., people from the museum). 

Discussions were conducted in class on the importance of using storyboards. Some local 
multimedia developers showed the storyboards they made, helping students to realize that 
storyboarding was a critical part of multimedia development. On several occasions during the 
semester, some students could not remember what to do next or what they had agreed upon 
initially. Storyboards became the blueprint with which they consulted to ensure they were on the 
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right track. When the deadline was getting closer toward the end of the semester and time could 
not be wasted, having a well-planned storyboard appeared to be particularly important. In 
addition, seeing the finished storyboards in a professional multimedia house and the CD 
programs based upon these storyboards helped students understand the significance of doing a 
storyboard and the necessity for doing it well. The comments by students such as "the main thing 
we [learned from visiting the multimedia company] was we need more storyboarding, planning 
out better before start" was a sharp contrast to their complaints and reluctance at the beginning of 
the semester. Moreover, storyboards became an efficient way to communicate their ideas with 
their clients from the museum. The results of this perception change illustrated the importance of 
providing an authentic design experience and apprenticeship to the students when designing a 
learner-as-designer environment. Simulating the classroom learning as a mini-multimedia 
production house and allowing students to interact with multimedia experts as well as their 
clients seemed to have contributed to this positive change. 

Collaboration Among Team Members 

Working with others who were different from oneself and working together in a team to 
accomplish a project in time are important aspects for successful multimedia development 
(McDaniel & Liu, 1996). Many of the students had strong personalities, and many did not know 
each other before the class. Although differences did exist, students in general exhibited their 
understanding of the importance of collaboration in a team; there was a significant increase in the 
design skill of collaboration from pretreatment to post-treatments as indicated in the ANOVA 
analysis. According to these students, "discussing with the team what information to include" 
and "brainstorming the best way to present an idea" were among the most important tasks. 
Students shared ideas and helped each other with multimedia tools, regardless of the teams. It 
was encouraging to note that many students, during the interviews, complimented on their 
teammates' work rather than their own. Although competitions existed among the teams, the 
students competed in a positive way, such as getting the project finished ahead of the other 
teams. 
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Such a positive atmosphere was partly attributed to the fact that the importance of 
collaboration among a team was emphasized by the teacher from early on. The teacher had 
assumed the role of a mediator for any conflicts. It was also attributed to the simulated 
multimedia production environment where each student had a distinctive role to play (whether it 
was a project manager for a team, a programm.er, an artist, or a researcher). Working in this 
environment helped students to see the importance of each role and to understand that the 
success of a project depended on whether a team could work together (McDaniel & Liu, 1996). 
Involvement in production allowed students to gain a first-hand experience in team collaboration. 

The complex nature of multimedia production gave students an opportunity to learn from 
each other's differences and develop the ability to solve conflicts. The students negotiated among 
themselves what role(s) to play and which task(s) to perform. The team project managers tried to 
ensure everyone was on task and motivate those who were not. Students began to exert peer 
pressure on those who tended to neglect the project. They solved the conflicts and differences 
through voting and group meetings. In short, designing multimedia programs provides students a 
meaningful learning environment in which group decisions and collaboration were exercised. 

Use of Multimedia Tools for Presentation 

Learning multimedia tools comprised a significant part of this learning experience. At the 
production phase, students began to u^e the multimedia tools to create what they set out to 
accomplish. Students divided the tasks among themselves: Some drew the pictures, some 
scanned the photographs, others made animations, and still others programmed. A topic for 
continuous discussion was how to present their ideas in an effective and interesting way. Since 
the audience for the three museum teams were mostly young children, it was important to make 
their programs both educational and entertaining. The Dinosaur program was game like: driving 
through a "Jurassic" like forest and into an ancient time when dinosaurs lived. The Physics team 
used a discovery approach for young children to find out how gravity worked. Students in the 
History team learned, on their own, a 3D modeling program to create a three-dimensional 
museum room for young users to browse. There were paintings on the wall that a user could click 
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and obtain further information of the painting and its representative historical age. To make their 
electronic version different from the print-based yearbook, the CD yearbook team used a variety 
of techniques (animation, video, audio, graphics and text) to represent many aspects of students’ 
life in a colorful and vivid way. 

In support of other research studies (Beichner, 1994; Heys, Weingard, Guzdial, Jackson, 
Boyle, & Soloway, 1993; Lehrer, et al., 1994; Spoehur, 1993), this study shows that multimedia 
tools have become a way to scaffold students' learning of design skills. Although the multimedia 
software used in this study allowed students the freedom to create what they desired, the students 
had the challenge of determining which media to use and for what purpose. They brainstormed, 
researched, and discussed how to best present their ideas. For those students who tended to sit at 
computers and create without planning, this exercise "forced" them to see the importance of 
careful selection of the media for the target audience and with a clearly defined objective. 

It is important to point out that many students had taken on the challenge of figuring out 
certain techniques for presenting information and then shared with others. The awareness of 
everyone being a learner including the teacher and other multimedia experts encouraged students 
to pursue independent and self-directed learning (Collins, Brown, & Newman, 1989). One 
student stated that "the teacher is not always teaching you, you teach the teacher and he helps 
you and you help him so everybody becomes a teacher." Judging from the limited time available 
in one semester, the technical skills these students developed and the fine quality of the 
accomplished projects were quite satisfactory according to the evaluation by the teacher, the 
museum people, and the graduate student (see Table 7). During an end-of-semester interview, all 
teams were asked to describe their projects, objectives, audience, choices of media, and 
techniques used. Students' mastery of the tools was obvious as they demonstrated and explained 
in detail how certain part was created. However, it was also clear that some students paid less 
attention to some of the details. For example, several screens were cluttered with many fonts and 
colors used without a clear purpose. The text on these screens was, therefore, difficult to read. 
Challenge of Designing for a Real Audience 
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One benefit of this "leamer-as-designer" environment was that students wor'ied closely 
with their clients (the Children's museum) and were creating programs for young children. 
Students' awareness of the needs of a target audience was expected to increase. The interview 
data showed that during the planning and design phases, the students had explicitly expressed the 
desire to satisfy the needs of their audience. For example, in presenting the concept of gravity to 
five-year-olds, students in the Physics group decided to use "the length of a football field" to 
explain the expression of "98 meters" as a distance of a falling object. They felt that for young 
children, the metaphor of a football field was more concrete and understandable than the concept 
of 98 meters. In another case, the students purposely used child-like handwriting to present the 
information in hope of making the program more attractive to young children. One student stated 
that "it is very important [that the kids like our program], because that's who we are making it 
for, and they are our target audience." However, the ANOVA results showed no change in the 
students' understanding of the "audience" (see Table 2). According to the results of the design 
task listing and ranking (which listed the tasks the students actually performed and the relative 
importance of each task), "testing" and "getting someone to try out the programs" seemed to be 
of low priority to the students (see Table 3). 

What appeared to happen was that although the students were aware of the needs of their 
target audience and the importance of satisfying them, the step of testing was neglected in the 
actual implementation due to the lack of time. Because testing by the target audience was the last 
step in the production process, it was not addressed when the time was insufficient toward the 
end of the semester and finishing the projects became the primary goal. Although informal 
evaluations occurred am.ong team members and between the teams, no systematic evaluation, 
especially evaluation by the audience, was performed. 

For many students, creating a program of their own ideas was very important. They 
perceived that multimedia design provided a way for them to be creative and express themselves. 
They developed a very strong sense of ownership for what they created. However, sometimes 
they were so involved in using their own ideas that they tended to forget about their audience. 
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When the students were asked if they took suggestions from the others (especially the clients), 
several of them replied that they would consider them if they liked them, not necessarily because 
the suggestions would be good for ihe intended audience. In other words, although the students 
were aware of the audience, their sense of ownership and individualization was strong. This 
finding suggests that it was not easy for the students to achieve a balance between designing for 
the intended audience and designing from their own ideas. They needed more assistance in 
performing the evaluation by target audience, especially when time is a factor. 

CONCLUSION 

This study found that the " learner- as-designer" environment described here had a positive 
impact on the at-risk high school students. As a result of participating in this project, the students 
showed a significant growth in their value of intrinsic goals. The experience helped students to 
acquire several critical design skills. The results of the study provided some supporting evidence 
to the concept of designers of knowledge (Perkins, 1986). What was most encouraging from the 
study was that this experience of designing multimedia programs for real audiences provided 
many students, who were considered at-risk and could not otherwise succeed in schools, a way to 
pursue their own goals. It helped them to search for their own identities and find their own 
meanings for learning. It also appeared that such an environment offered a promising opportunity 
for students to exercise and develop their higher order thinking skills. 

Although the results of what the students have accomplished are encouraging, we face 
many more challenges. Designing multimedia programs by students offers a new way of learning 
and teaching. Research in this area has just begun. As Carver and her colleagues point out 
(1992), the challenge is for us to continue to search for ways of designing learning environments 
that foster knowledge constmetion. 
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Table 1 . Mean and Standard Deviation (in Parenthesis) for the Motivation Scores 





Intrinsic 


Extrinsic 


Task Value 


Belief 


Self-Efficacy 




Pre 


Post 


Pre Post 


Pre 


Post 


Pre 


Post 


Pre 


Post 


Control G 


5.3 

(1.4) 


5.3 

(1.3) 


4.8 4.6 

(1.2) (1.3) 


5.9 

(1.2) 


5.6 

(1.3) 


5.4 

(1.0) 


5.4 

(1.1) 


5.1 

(1.3) 


5.0 

(1.7) 


Treatment G 


5.2 

(.8) 


6.3* 

(.6) 


5.2 5.4 

(1.1) (1.3) 


5.8 

(.9) 


6.5* 

(.6) 


5.6 

(.8) 


5.8 

(1.0) 


5.5 

(.8) 


6.5* 

(.5) 



* = significantly different from the control group, P<.05. 
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Pre 



Post 



Intrinsic goal orientation scores 




Task value scores 




Self-efficacy scores 

Figure 1 , Scores for treatment and control groups from pre to post. 
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Table 2. Mean and Standard Deviation (in Parenthesis) for the Design Skills 



Planning 


Searching 


Presenting 


Connecting 


Audience 


Collaboration 


Mental 

Effort 


Interest 


Individual 


Pretest 4.69 


4.66 


5.03 


4.4 


5.58 


4.7 


4.29 


5.31 


5.0 


(.83) 


(.8) 


(.86) 


(.94) 


(1.3) 


(.94) 


(.89) 


(1.0) 


(.83) 


Posttest 5.49* 


4.78 


5.84* 


4.96 


6.0 


5.62* 


4.84* 


6.13* 


5.0 


(.89) 


(.78) 


(.88) 


(1.0) 


(1.13) 


(1.18) 


(.82) 


(1.0) 


(1.1) 



* = significantly different from the pretest, P < .01. 



O 

ERIC 
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Table 3. Frequency of Design Tasks Mentioned by the Students 



Tasks Freq. Examples of Helpful Strategies Examples of Obstacles 

Listed by the Students Listed by the Students 



With highest Freq. 

Doing Storyboard 88% 

Researching 75% 

Programming 63% 

Brainstorming 56% 

Creating Graphics 56% 



• "working in groups" 

• "ideas from the group" 



"librarian and school library" 
"getting information on sports 
from coaches" 

"me and B worked together" 
"got help on Director from 
other people" 

"We all threw out a bunch of 
ideas." 

"working in groups" 

"A is an excellent artist. We 
all did graphics though." 

"J gave me hints and clues for 
tools in Photoshop . D made 
suggestions." 



• "took a lot of production time' 

• "we could not understand 
why we needed storyboai’d" 

• "boring" 

• "books did not contain all the 
information we needed" 

• "computer locked up" 

• "crashing, crashing, and 
cashing, redid 3 times" 

• "We came up with too much.' 

• "time" 



"time and not knowing 
programs well enough." 
"time" 



With least Freq. 

Scanning Pictures 19% 

Digitizing Voice 19% 

Making Background 13% 

Digitizing Music 6% 

Taking Photos 6% 



"teacher taught me how to use 
the scanner." 

"I did the voice and all the lines. 
"I did background myself.” 

"M brought CDs." 

"D helped me." 
weather." 



• "scanner too slow" 

• "everybody wasn't being quiet!!! 

• " not enough people to help 
draw." 

(no example) 

• "It was very cold and rainy" 



Testing 



6 % 



(no example) 



(no example) 
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Table 4. Time Spent In and Out of the Class 


Time Spent 


Based Upon Observation and Intetyiew Data 


Earlier in the Semester 

In Class 


• About half of the class were on task. 

• From time to time, quite a few students were hanging around in 
classroom, talking to other students (not a pai't of the class), or doing 
nothing. 

• Teacher called upon students to be on task. 


Out of Class 


• CD Yearbook team met weekly in Denny's to discuss the project and 
socialize. For other groups, not much was going on. 


Later in the Semester 

In Class 


• During most of the observed periods, students were sitting in front of the 
computers working. Some sat together providing feedback. Some were 
discussing about the project in a circle. Most were on task. 

• Museum people' s comment: "When we first started the project, 
[students] tended to just sit there and not be actively involved. It seems 
to me that they changed in the course of the semester. vVhen one of 
them was going to the library, others in the group would say 'Will you 
check to see if they are on task.' So they started policing each other. 
None of them would have done that at the first of the semester." 


0 


• Museum people' s comment: "At first, as little as possible [working on 
the project]. Then 40 or 50 percent by about half way [of the semester]. 
Now I'd say most of the groups are on task throughout the entire class." 


Out of Class 


• About half of the class spent their lunch time regularly working on the 
projects. A third of the class spent the morning or afternoon period 
working on their projects. Some spent both. During the spring break, 
about 50% of the class showed up on the day they talked their teacher 
into opening the computer rooms. 

• Student's comment: "I spend about 5 hours a day on this project. I have 
first and third periods, my whole lunch period, before school and 
sometimes after school." 

• Museum people's comment: " They come after class, they come on 
weekends, they take it home and work on it." 

• Teacher's comment: "For example, the dinosaur group. They came in 
extra a lot. I stayed with A until 6:30pm one day and gave him a ride 
home. A stayed on his own a couple of other times. B would come early 
or he would stay late. Then C came in early quite a few times. In fact 
that day when the school was out and she has graduated, she called me at 
6:40 in the morning wanting to know when I could take her up to school 
so she could work [on the project] until the time she has to go to work 
[to earn living]." 
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Table 5. Students' comments about their abilities and confidence 



Categories 


Students' statements 


About professionals' 
feedback to students' 
programs 


• We got a chance to show them our stuff as well. They 
were impressed. 

• They said they liked it a lot. 

• They said they liked the physics group, the screens, the 
layout. They liked the whole. 

• They liked the CD control panel B made. Theirs is a little 
more advanced, but they liked ours. 

• They liked that one [the dinosaur project] a lot, thought 
that was funny. 


About confidence in one's 
own ability 


• We have a good head start. They [professionals] wish they 
had what we have. 

• We have basically the same [tools, comparing 
professionals' work with their own], but they just put 
more time into it. They've had more time and experience. 
But other than that they are pretty much the same. 

• I think the only difference is experience, really and truly. 

• They have a lot more time than we have to do our stuff. 
That's one of the main factors. But we use some of the 
same programs. 


About desire to be in 
multimedia profession 


• We liked what they were doing. Everybody wanted to go 
there again. 

• We liked their lab, laid out in such a way that you can get 
a lot of intense work done. 

• I'm amazed by what they were doing. I want to work 
there. 

• To continue making interactive CDs [after graduating 
from high school], and doing a lot of filming of movie". 

• I enjoyed the whole thing [the crip]. That is exactly what 1 
want to do when I get out of high school. 
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Table 6. Collaboration in the Team 


Categories 


Students' Statements 


Sharing in a team 


• Right now we are working independently, but we all come together for group 
meetings. 

• We ai'e all going to share the burden [scanning several hundreds of photos]. 
When you are working on a task of such scale, there's gonna [to be]a lot of 
tasks that are monotonous so we're all going to have a bite [of] the bullet. 

• I just got back from the other room where we were discussing our different 
ideas. The other four people have been working together in the other room 
while I was working on this [a major interface for the project]. 

• [How often do you talk as a group?] Just about everyday. 


Helping each other 


• B's helping S and J's scanning. We're all working [together]. 

• Though only one of us can be on the computer at a time [referring working 
at home], we are all chewing on ideas. That looks terrible, that looks great. 
It is neat. It has been interesting to see how easily we come to consensus. 

• [Where did you learn this technique?] M had used it with his program. He 
showed me. 


Appreciating other's 
work 


• The Physics group is doing stuff on falling objects. That's pretty neat. 

• M can do Director great. 

• Like S, he did a real nice QuickTime thing. He made screen actually coming 
towards you. He's done a real good job. 


Taking others' 
suggestions 


• J and myself were involved in programming to put imported 
pictures together. Other people would give their ideas on how we could 
make it easier and to get the project done faster. We would think about 
them. We would use them if we thought it would speed things up. 


Deciding together 


• We all decide together. 

• We [CD yearbook group] all brainstoiTned. It [giving out surveys to students in 
the school] was more of a group idea. 

• We put up what areas there are and let people choose where they want to be. 

If there happens to be too many people in one area and not enough in another 
we just tell people and ask for someone to volunteer to work in that area. So 
far that has worked. 


Getting along with 
each other 


• [Do you think your group gets along well?] Yes. Everything's been worked 
out. When people have different opinions, we just work it out, talking to each 
other. We take votes and stuff. 

• [From the graduate student] I mean there have been some conflicts, but then I 
think that if there are no conflicts, people are not really thinking. Sol have 
been happy with how they have worked together. I remember one day A, B 
and I were working on the program and it was a really difficult task. We were 
all really into it. We were all debating about the best way to proceed and we 
got a little hot under the collar. Then we stepped back and said 'Hey, look at 
what we are working on. Let's just step back and do some flowcharts and see 
what is the best way to proceed.' So it worked out. And there have been 
several other little conflicts like that, people yelling at each other. And we just 
step back and said we are all a team. "There is nothing to make enemies about. 
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Exerting peer pressure • [From museum people] They started exerting peer pressure on each other for 

those who were not holding up their end of the work. Like saying 'well, if we 
could just get so and so to work' and they would actually try to. They would 
go through their storyboard and see what they don't have and then try to get 
someone who is not working to do that for them. 'We need this.' They say 'we.' 
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Table 7. Evaluation of Students' Technical Skills 



Perspectives 


Comments 


From the teacher 


• Some [students] are far beyond me, when it ccmes to 
Director. 


From the museum people 


• I'm ecstatic v;ith their work. Their work is excellent. ... 
I'm very happy with their work. I think it is great. It is 
practically production quality in one semester. Yes, I'm 
real happy with them. 


From the graduate student 


• They picked up the programs [i.e. software] very quickly. 
Everyone in the group has produced something that they 
can show. I am very happy with what they have 
accomplished... 


From the students 


• We all learned more or less Director. I mean, at the 
beginning of the semester, nobody knew how to turn on 
Director. Onlv a few people did. But compared to now, 
everybody more or less in this class has the ability to 
write their own Director movie. Thev might not be the 
fanciest thing, but everybody can write some lingo, know 
how to put the things together, and how to import 
graphics. That is something. 

• Evervbodv in here is verv familiar with Director interface, 
and has the potential to keep on learning it. 


From visitors at the open 
house 


• Great! You cannot tell this is produced by high school 
students. That's very encouraging. 
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Figure 4. Screens of the Programs Students Developed 




CD year book: Student life section 




History program: A 3D exhibit room 




Dinosaur program: Main menu 




Physics program: Main menu 
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